Dynamic Shader Lamps: Painting on Movable Objects

We present a Dynamic Spatially Augmented Reality system for augmenting movable 3D objects in an indoor environment using multiple projectors. We describe a real-time system for applying virtual paint and textures to real objects simply by direct physical manipulation of the object and a “paint brush” stylus. We track the objects and the “paintbrush”, and illuminate the objects with images that remain registered as they move, to create the illusion of material properties. The system is simple to use and we hope it may herald new applications in diverse fields such as visualization, tele-immersion, art and architecture. The system currently works with tracked objects whose geometry was pre-acquired and models created manually, but it is possible to extend it, by adding cameras to the environment, to acquire object geometry automatically and use vision-based tracking for the object and paintbrush.